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ABSTRACT 

A  need  for  a  high-fidelity  sensor  design  simulation  model  to  accurately  predict  the  envelope  of  the  system 
performance  and  to  offset  the  escalating  cost  of  the  system  development  and  testing  are  widely  accepted  by  the  defense 
community.  This  paper  presents  one  such  example  of  the  modeling  capability  developed  for  the  Ballistic  Missile  Defense 
(BMD)  application,  called  the  Signal  Processing  Environment  for  Analysis  and  Reduction  (SPEAR)  simulation. 

SPEAR  has  become  a  key  IR  sensor  design  and  signal  processing  performance  verification  tool  for  the  BMD 
Advanced  Sensor  Technology  Program  (ASTP),  the  Discriminating  Interceptor  Technology  Program  (DITP),  and  the 
Ground  Based  Interceptor  (GBI)  and  ,  where  it  is  used  for  sensitivity  analyses,  algorithm  evaluations,  and  performance 
assessments.  Eor  these  programs,  SPEAR  provides  an  algorithm  testing  simulation  to  evaluate  candidate  signal  processing 
options,  and  implement  and  test  performance  of  algorithms  proposed  through  advanced  technology  programs.  In  addition, 
SPEAR  is  used  to  process  real  world  data  to  provide  assessments  of  sensor  performance  and  provide  preflight  predictions. 
The  simulation  has  been  interfaced  to  the  Synthetic  Scene  Generation  Model  (SSGM),  a  community  standard  background 
and  target  scene  generation  simulation.  Through  this  interface  sensor  performance  can  be  evaluated  against  realistically 
modeled  backgrounds  to  evaluate  filtering,  detection,  and  false  alarm  performance. 

SPEAR  is  a  hi-fidelity  passive  infrared  (IR)  sensor  and  signal  processing  simulation  for  staring,  scanning,  and 
hybrid  sensors.  It  allows  the  user  to  specify  the  IR  sensor  physics  including  the  sensor,  optics,  focal  plane  array  or  scan  chip 
assembly,  analog  signal  processor,  time  dependent  and  object  dependent  processing  parameters  and  specific  noise  sources 
such  as  optics,  jitter,  fixed  pattern  noise,  dark  current,  and  gamma  spike  noise.  SPEAR  is  an  Ada  /  PVWAVE  combination. 
The  sensor  and  signal  processing  is  written  in  Ada  and  the  execution,  parameter  input,  and  function  analysis  are  controlled 
with  the  graphical  user  interface  (GUI)  written  in  PVWAVE.  The  signal  processing  techniques  available  as  options  include 
time  dependent  processing  techniques  such  as  adaptive  threshold  detection,  background  estimation  and  removal, 
morphological  filtering,  match  filtering,  target  signature  extraction  and  object  dependent  processing  techniques  such  as 
centroiding  and  pulse  matching.  SPEAR  has  simulation  control  options  to  allow  the  user  to  execute  and  examine  data  per 
frame  (mission  mode)  or  in  a  statistical  mode  to  investigate  parametric  sensitivities  of  the  sensor  performance. 
Documentation  of  SPEAR  includes  manuals  on  the  GUI,  the  SPEAR  application  components,  and  guidelines  for  adding  new 
algorithms  and  features. 

This  paper  provides  a  summary  of  key  algorithm  and  options  in  SPEAR.  Examples  of  performance  analysis  results 
are  provided.  The  paper  includes  stochastic  analyses  of  both  the  above-the-horizon  and  below-the-horizon  engagements  of 
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target  and  background  generated  scenes  using  SSGM.  Also  discussed  are  the  evaluation  of  radiometric  measurement 
precision,  angular  measurement  precision,  and  detection  of  targets  of  varying  intensities  with  respect  to  varying  sensor  signal 
processing  techniques. 


1.  SPEAR  OVERVIEW 

SPEAR  is  both  an  environment  for  simulation  of  IR  sensors  and  signal  processors  as  well  as  a  functional  analysis 
tool,  for  characterizing  deterministic  signal  processor  performance.  This  dual  nature  allows  the  analyst  to  model  and  predict 
the  behavior  of  a  sensor/signal  processor  combination  and  thereby  to  optimize  the  system  design.  Just  as  important,  it  allows 
the  analyst  to  process  actual  measurement  data  taken  from  a  sensor  system  to  perform  data  assessments.  Having  the  ability 
to  process  data  from  the  systems  being  modeled  provides  a  high  level  of  confidence  in  the  accuracy  and  fidelity  of  the 
simulation. 

The  SPEAR  application  components  are  implemented  as  modular  functional  models  of  threat  and  background 
scenes,  IR  sensor  components,  and  sensor  processing  algorithms  written  in  Ada,  approximately  50k  lines  of  code  (LOG). 
SPEAR  is  a  mature  simulation  and  has  become  a  key  sensor  simulation  which  has  evolved  through  the  following  programs: 
EAS  /  AOA  /  AST  /  GSTS  /  BP  /  BE  /  GBI  /  ASTP  /  DITP. 

The  Graphical  User  Interface  (GUI)  is  written  in  PVWAVE,  consisting  of  40k  LOG  and  is  very  easy  to  use.  It 
controls  the  set  up  and  maintains  all  inputs  and  allows  data  visualization  at  every  level  for  simulation  execution,  sensor  and 
signal  processing  parameter  setup,  and  functional  performance  analysis. 

The  generic  SPEAR  simulation  has  the  capability  to  model  either  a  staring  or  scanning  sensor  system.  The 
modularity  of  the  SPEAR  implementation  allows  for  other  types  of  sensor  systems  to  be  easily  implemented.  SPEAR 
provides  hi-fidelity  modeling  of  the  effects  of  the  targets  and  background  at  the  aperture  through  the  optical  train  to  the  Eocal 
Plane  Array  (EPA)  and  then  processed  through  the  Analog  Signal  Processor  (ASP),  including  the  noise  effects  through  the 
process.  The  data  is  then  sent  to  the  Time  Dependent  Processor  (TDP),  so  named  because  its  performance  requirements  are 
driven  by  the  sample  rate  of  the  sensor,  not  by  the  characteristics  of  the  target  scene.  The  TDP,  operating  on  all  the  EPA 
samples,  contains  algorithms  to  enhance  the  signal-to-noise  ratio,  isolate  regions  of  interest  in  the  scene  and  package  the 
selected  data  to  be  sent  to  the  Object  Dependent  Processor  (ODP).  The  performance  requirements  of  the  ODP  are  driven  by 
the  number  of  possible  objects  that  require  processing.  The  job  of  the  ODP  is  to  determine  the  precise  position  and 
amplitude  of  the  targets.  The  resulting  data  is  packaged  into  object  sighting  messages  on  a  frame-by-frame  basis  and  entered 
into  track  files.  In  the  case  of  an  interceptor,  the  track  files  are  used  onboard  to  guide  the  vehicle  to  an  intercept.  The 
diagram  below  shows  the  flow  of  data  from  the  threat  /  environment,  through  the  signal  generator,  through  the  signal 
processor  functions,  and  output  to  the  data  processor. 
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FIGURE  1.  SPEAR  FUNCTIONAL  FLOW 

SPEAR  follows  this  functional  flow,  with  emphasis  on  the  signal  processor.  There  are  many  processes  within  each 
major  function  that  the  analyst  can  set  up  to  reflect  the  concept  under  study.  If  actual  ground  test  or  flight  data  is  available, 
the  data  can  be  injected  into  the  TDP  or  at  the  interface  between  the  TDP  and  ODP,  depending  on  the  type  of  data  available. 


SPEAR  provides  a  flexible  model  of  a  sensor  system  to  drive  the  signal  processor  when  controlled,  simulated  data  is  desired. 
When  using  simulated  sensor  data,  Monte  Carlo  runs  can  be  made,  varying  noise  draws  on  each  trial  and  accumulating 
statistics  on  performance.  Typical  performance  measurements  include  Probability  of  Detection,  Probability  of  False  Alarm, 
Radiometric  Measurement  Precision  (RMP),  Angular  Measurement  Precision  (AMP),  and  probability  of  resolution  for 
Closely  Spaced  Objects  (CSOs). 

A  fundamental  strength  of  SPEAR  is  that  the  analyst  can  change  the  sensor  parameters  or  the  signal  processor  setup 
without  re-compiling.  All  changes  are  made  through  the  GUI  with  mouse  clicks  and  minimal  typing.  Extensive  plotting 
capabilities  are  included  and  these  also  are  controlled  by  the  GUI.  These  attributes  have  led  to  a  tremendous  increase  in 
productivity.  The  analyst,  who  needs  very  little  computer  expertise,  can  study  more  concepts  in  a  shorter  period  of  time  with 
much  less  opportunity  for  error. 


2.  SPEAR  EXECUTION 

The  SPEAR  GUI  main  window  shown  in  Figure  1,  displays  the  top  level  user  options.  The  user  operates  within  a 
Project,  which  groups  all  the  inputs  and  outputs.  The  Project  can  be  saved,  new  ones  can  be  created,  and  existing  Projects 
can  be  modified,  all  using  the  GUI.  This  method  of  controlling  the  setup  and  execution  has  two  important  benefits:  reducing 
the  likelihood  of  user  error,  and  reducing  the  time  required  to  setup  and  document  results. 

SPEAR  can  be  run  in  two  basic  modes:  Mission  and  Monte  Carlo.  In  the  Mission  mode,  one  or  more  frames  of 
target  data  is  set  up  (or  read  in  from  an  external  source)  and  each  frame  is  processed  once.  In  the  Monte  Carlo  mode,  a  single 
frame  is  executed  many  times  while  varying  noise  parameters  for  each  frame  which  allows  for  the  accumulation  of  statistics 
for  algorithm  performance. 


FIGURE  2.  SPEAR  MAIN  WINDOW 


The  Project  is  composed  of  seven  “objects”  that  contain  the  setup  parameters.  To  edit  the  objects,  the  user  clicks  on 
the  object  icon,  creating  new  screens  and  dialog  boxes  for  setting  up  the  parameters.  The  setup  objects  are  described  below: 

2.1  Threat  Definition 

The  threat  definition  window  represents  the  sensor  field  of  view  with  a  representation  of  virtual  pixel  locations 
where  the  position,  amplitude,  and  velocity  of  point  source  targets  and  extended  source  targets  can  be  can  be  easily  created, 
moved,  duplicated,  and  deleted  with  a  graphical  interface.  The  user  also  has  the  option  to  setup  multiple  frames  of  target 

data  to  perform  mission  mode  analyses.  The  target  amplitudes  can  be  input  in  units  of  irradiance  (W/cm^)  or  a  signal-to- 
noise  ratio  (SNR),  based  upon  the  noise  equivalent  input,  for  up  to  3  wavebands.  The  last  option  of  the  threat  model  is  to 
allow  the  user  to  specify  the  exact  locations  of  the  targets  and  their  mappings  to  the  FPA  or  the  user  can  specify  to  bound  the 
specified  targets  in  the  field  of  view  and  map  them  to  the  center  of  the  FPA. 
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FIGURE  3.  THREAT  SETUP  WINDOW 


2.2  Noise  Source  Models 

The  noise  module  includes  seven  types  of  noise  sources  which  consist  of  white  noise,  background  noise,  jitter, 
optical  noise,  fixed  pattern  noise,  dark  current,  and  gamma  noise.  The  noise  sources  are  calculated  for  each  detector  in  the 
user  specified  FPA.  There  are  other  noise  terms  which  may  be  important  to  consider  based  upon  the  sensor  the  user  is 
evaluating  and  these  models  can  be  easily  implemented  into  the  modular  architecture  of  SPEAR. 

White  Noise.  The  white  noise  model  is  modeled  as  random  draws  from  a  gaussian  distribution  with  the  mean  of  zero  and  the 

standard  deviation  equal  to  the  noise  equivalent  input  in  units  of  W/cm^.  The  user  can  set  the  noise  seed  for  the  random 
number  generator  and  can  choose  to  specify  the  random  numbers  used  in  each  calculation  to  be  read  in  from  an  external  file, 
which  is  useful  for  execution,  repeatability,  and  verification  of  large  mission  analyses  where  many  frames  of  data  are 
processed  in  subsets. 


Background  Noise.  The  background  noise  model  consists  of  specifying  a  previously  generated  scene  utilizing  the  Synthetic 
Scene  Generation  Model  (SSGM),  a  government  supplied  model,  sponsored  by  the  Ballistic  Missile  Defense  Organization, 
and  developed  by  the  Naval  Research  Laboratory.  For  analysis  requiring  an  SSGM  background  scene  as  input  requires  the 
user  to  specify  the  sensor  model  parameters,  target  parameters  (whether  in  boost  phase  with  plume  effects  or  post  boost 
phase),  time  of  day,  viewing  geometry,  and  the  corresponding  terrain,  cloud,  or  celestial  backgrounds  (in  some  cases  the 
scenes  may  contain  all  three,  for  a  sensor  viewing  both  below  the  horizon  and  above  the  horizon).  SSGM  generates  the 

background  scene  in  radiance  units  of  W/cm^/sr  at  the  sensor  aperture.  SPEAR  converts  the  input  background  scene 
radiance  at  the  aperture  into  irradiance  at  the  aperture.  The  user  has  the  option  to  generate  a  background  scene  without 
embedded  targets  so  that  the  targets  can  be  placed  in  specified  regions  of  interest  by  using  the  threat  model  specified  above. 
This  option  allows  the  user  to  either  use  the  entire  background  scene  as  an  input  or  specify  a  specific  sub-region  of  interest  or 
randomly  select  regions  to  generate  performance  statistics. 


Jitter.  The  jitter  model  is  used  to  model  the  telescope  jitter  by  combining  user  defined  sine  waves  imposed  on  the  line  of 
sight.  The  user  specifies  the  number  of  sine  waves,  their  amplitudes,  and  their  frequencies. 

Optical  Noise.  The  effective  optical  photon  flux  from  the  telescope  components  is  set  up  here  by  specifying  the  waveband, 
temperature,  and  emissivity  of  the  elements.  Calculation  of  the  exitance  of  the  optical  elements  is  performed  by  integrating 
the  Plank  blackbody  equation  over  the  waveband(s)  of  interest  for  the  specified  temperature.  The  specified  emissivity  is 
used  to  calculate  the  radiant  intensity  of  the  optical  elements.  The  mean  optical  background  flux  level  is  converted  from 
radiant  intensity  to  electrons  and  the  standard  deviation  is  calculated  as  the  square  root  of  the  mean  electron  flux  level.  The 
user  specifies  the  random  number  seed  for  the  noise  fluctuation  which  is  modeled  as  random  draws  from  a  gaussian 
distribution  with  the  specified  mean  and  standard  deviation. 

Fixed  Pattern  Noise.  The  fixed  pattern  noise  model  models  the  detector  non-uniformity  for  each  detector  in  the  array.  Here 
the  user  specifies  the  percent  non-uniformity  across  the  array  which  is  modeled  as  random  draws  from  a  gaussian  distribution 
with  the  mean  equal  to  the  specified  effective  offset  value  and  the  standard  deviation  equal  to  the  percent  non-uniformity 
times  the  mean  value.  The  modeling  of  the  responsivity  (gain)  of  the  detectors  is  in  progress. 

Dark  Current.  Dark  current  is  the  current  generated  from  the  detector  when  it  is  non-illuminated.  The  user  can  specify  the 
effective  dark  current  and  the  noise  seed  value.  For  each  pixel  in  the  array  the  model  calculates  the  effective  dark  current  by 
random  draws  on  a  gaussian  distribution  with  the  mean  equal  to  the  dark  current  and  the  standard  deviation  calculated  within 
the  model. 

Gamma  Noise.  The  gamma  noise  model  is  used  for  simulating  the  effects  of  natural  space  (Van  Allen  belt)  environment  or 
nuclear  bursts  on  the  sensor  and  signal  processor.  The  user  can  select  the  pulse  height  distribution  file  to  be  used  and  the 
random  number  seed. 

2.3  Sensor  Design  Parameters 

In  this  dialog  box,  the  basic  sensor  parameters  are  defined.  The  type  of  sensor  is  defined,  along  with  the  aperture 
diameter,  the  frame  time,  the  sample  time,  number  of  samples  per  dwell,  the  over-sample  rate,  the  degree  of  gamma 
shielding,  the  energy  focused  on  detector,  the  noise  equivalent  input,  the  optics  transmission,  the  quantum  efficiency,  and  the 
waveband(s)  of  interest. 
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FIGURE  4.  SENSOR  SETUP  WINDOW 


2.4  Focal  Plane  Array  (FPA)  Design  Parameters 

Here  the  user  sets  up  the  focal  plane  array  design  parameters.  The  dialog  box  changes  to  reflect  the  type  of  sensor 
defined  in  the  sensor  dialog  box.  If  the  sensor  is  staring,  the  user  selects  the  number  of  rows  and  columns  of  the  mosaic. 
The  number  of  columns  defines  a  Time  Delay  and  Integration  (TDI)  set  for  a  scanning  sensor.  The  relative  sensitivity  of 
each  detector  column  is  set  here.  For  either  type  of  sensor,  the  user  specifies  the  physical  and  angular  dimensions  of  each 
detector,  the  “deadspace”  between  detectors  and  groups,  detector  filter  transmission,  coupling  factor,  blocking  factor, 
capacitance,  and  the  gain  term.  SPEAR  calculates  the  amount  of  energy  reaching  each  pixel  in  each  waveband  and  maps  it 
to  the  FPA  using  the  internally  generated  gaussian  detector  response  function  or  an  externally  generated  response  function 
from  an  optics  model  like  CODE  -  V. 


FIGURE  5.  FPA  SETUP  WINDOW 
2.5  Analog  Signal  Processor  (ASP)  Modules 

Inside  the  ASP  dialog  box  the  user  can  select  from  a  list  of  functions  performed  in  the  ASP.  These  functions 
include  analog-to-digital  conversion  (A/D),  background  filtering,  gamma  mitigation,  Nyquist  rate  restore,  and  sum 
oversamples.  For  the  A/D  conversion  the  user  specifies  the  maximum  range  of  the  A/D  (e.g.  for  a  12  bit  A/D  the  maximum 

range  is  calculated  as  2^^  -  1  =  4095  counts),  and  the  user  specifies  the  voltage  range  across  the  A/D. 


2.6  Time  Dependent  Processor  (TDP)  Modules 

This  is  the  dialog  box  most  frequently  used  by  the  SPEAR  analyst  (see  Eigure  6).  Here  the  user  defines  the 
sequence  of  functions  for  the  TDP.  The  user  selects  from  the  “Available  Eunctions”  list  and  inserts  it  in  the  processing  chain 
by  pointing  and  clicking.  SPEAR  does  a  considerable  amount  of  work  behind  the  scenes  to  ensure  that  the  algorithms  will 
work  together  properly  even  with  the  changes  in  the  sequence. 

If  the  analyst  is  using  data  imported  from  another  source  (e.g.  sensor  test  or  flight  data),  the  “interject  data”  button  is 
selected  and  the  arrow  is  placed  to  show  at  what  point  in  the  chain  the  data  is  to  be  inserted.  In  some  concepts,  data  is 
filtered  to  improve  signal  to  noise  ratio  for  detection  purposes,  but  unfiltered  data  is  used  for  accurate  determination  of  target 
position  and  amplitude.  SPEAR  allows  the  user  to  specify  at  what  point  the  unfiltered  data  is  to  be  saved  for  position  and 
amplitude  determination.  The  data  is  then  processed  by  extraction  stripping  to  extract  the  regions  above  threshold. 


FIGURE  6.  TDP  SETUP  WINDOW 


The  TDP  setup  dialog  box  is  where  the  modular  structure  of  SPEAR  is  most  apparent  and  where  the  benefits  of  the 
Ada  language  are  demonstrated.  The  functions  can  be  inserted,  deleted,  modified,  or  re-ordered,  mostly  by  pointing  and 
clicking.  Typically  this  can  be  done  in  only  a  few  seconds  and  the  case  can  be  re-run  without  having  to  re-compile  the 
simulation.  For  example,  if  the  processor  appears  to  be  passing  too  many  false  alarms  or  low-amplitude  targets,  the  detection 
threshold  can  be  raised  as  desired.  Or  a  background  filter  can  be  selected  that  provides  more  clutter  suppression.  This 
allows  very  rapid  turnaround  in  evaluating  algorithm  performance  or  in  analyzing  data.  If  the  user  is  interested  in  evaluating 
a  signal  processing  algorithm  which  is  currently  not  implemented  into  the  TDP,  the  modular  design  of  the  TDP  allows  for 
easy  implementation  and  testing. 

2.7  Object  Dependent  Processing  (ODP)  Modules 

The  dialog  box  for  the  ODP  (see  Figure  7)  is  where  the  analyst  determines  how  the  position  and  amplitude  are 
estimated.  The  basic  choices  are  Pulse  Matching,  Centroiding,  and  Super  Resolution  with  choices  for  parameters  appearing 
(or  being  hidden)  in  the  box  based  on  which  method  is  selected. 

The  example  in  Figure  7  shows  the  Pulse  Matcher  selected.  The  Figure  of  Merit  (FOM)  factor  determines  how 
“target-like”  a  detection  has  to  be  for  it  to  be  declared  a  true  target.  The  initial  classification  button  brings  up  a  sub-window 
where  the  analyst  can  select  a  screening  method:  second  moments  or  a  simple  size  check.  The  buttons  for  color  correlation 
and  re-correlation  are  dimmed  because  only  one  waveband  is  being  processed  in  the  current  setup.  If  multiple  wavebands 
had  been  selected,  this  option  would  direct  SPEAR  to  attempt  to  associate  target  sightings  from  individual  bands,  thus 
reducing  the  false  alarm  rate.  The  Image  Motion  Parameter  Compensation  (IMPC)  is  a  method  for  mitigating  the  effects  of 
target  motion  across  the  focal  plane.  Clump  processing  and  fine  partitioning  are  two  methods  for  dealing  with  multiple 
targets  that  are  so  closely  spaced  that  they  extend  across  many  detectors.  Additional  choices  appear  when  these  options  are 
selected  for  setting  sup-parameters. 

The  Aikaike  Information  Criteria  (AIC)  is  used  in  the  decision  process  of  classifying  the  pulse  matcher  fit  of  the 
extracted  data  packet  as  either  containing  one  or  two  targets.  The  Eta  parameter  in  the  AIC  equation  is  a  weighting  factor  in 
the  decision  process  and  provides  a  method  of  fine  tuning  the  pulse  matcher  performance. 


Resolution  Method:  Pulse  Matching 

^  Super 

Resolution  ^  Centroid 

TDP  data  format: 

Strips 

J 

Auto  Calc  Step  Size:  ^  Off  ^  On 

FOM  factor: 

100.000 

J 

Single:  0.200000  |  Dual:  0.200000  | 

PM  Iterations: 

20 

J 

Convergence  Criterion: 

S.SOOOOe-05 

J 

Eta  For  AIC: 

15.0000 

1 

Initial  Classification: 

^  Off  ^  On 

1  Set.. 

* 

’V'  =■■==  =  =■-  = 

IMPC: 

^  Off  ^  On 

Fine  Partitioning: 

^  Off  ^  On 

j  OK  1  Cancel  | 

Help 

FIGURE  7.  ODP  SETUP  WINDOW 
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FIGURE  8.  EXECUTION  SETUP  WINDOW 


2.8  Execution  Modes 

Once  the  setup  is  completed,  the  user  pushes  the  “Execute”  button  and  the  execute  dialog  box  shown  in  Figure  8 
appears.  If  in  the  mission  mode,  the  execute  window  will  allow  the  user  to  pick  a  range  of  frames  for  processing,  what 
intermediate  data  to  save,  whether  to  use  the  default  Gaussian  reference  pulse  or  an  externally  supplied  reference  pulse  from 
CODE  -  V  for  the  detector  response  function  and  pulse  matching,  whether  to  use  or  ignore  the  target  velocity,  and  whether  to 
vary  the  target- to-detector  phasing.  The  execute  dialog  box  is  also  where  the  options  for  injecting  data  are  chosen  and  where 
the  execution  options  for  execution  and  scoring  are  chosen. 

If  the  user  selects  the  Monte  Carlo  mode  in  the  SPEAR  main  window  prior  to  execution,  a  list  of  parameters  appears 
that  will  be  the  “independent  variable”  in  the  Monte  Carlo  analysis.  Items  are  added  and  deleted  in  a  manner  similar  to  the 
TDP  setup  window.  When  the  “Execute”  button  is  pressed,  the  user  selects  the  number  of  trials,  along  with  the  normal 
settings  described  above. 


3.  FUNCTIONAL  PERFORMANCE  ANALYSIS 

The  user  may  perform  two  types  of  analyses:  1)  Mission  Mode,  or  2)  Statistical  Analyses  Mode.  The  first.  Mission 
Mode,  is  used  for  assessment  of  signal  processor  performance  per  frame  of  data.  Processing  algorithm  options,  including 
background  filtering  techniques  can  be  applied  to,  for  instance,  assess  the  ability  of  the  signal  processor  to  detect  and  extract 
the  target.  Operation  in  the  mission  mode  is  well  suited  for  predicting  sensor  preflight  performance  over  the  multiple  frames 
during  the  mission  and  for  evaluating  the  sensor  postflight  data  to  verify  and  validate  SPEAR  modules.  The  second  type  of 
analyses.  Statistical  Analyses,  is  used  to  obtain  a  statistical  evaluation  of  sensor  measurement  performance.  In  this  mode,  a 
Monte  Carlo  approach  is  used  to  generate  a  large  number  of  realizations  of  a  target  against  the  defined  background  to 
provide  evaluation  of  sensor  figures  of  merit,  including  probability  of  detection  and  probability  of  false  alarm. 


3.1  Mission  Mode  Analysis 

Example  results  for  Mission  Mode  analyses  cases  are  shown  in  Figures  9,  10,  11.  When  the  user  clicks  the  “TDP 
Analysis”  button  the  user  sees  the  algorithm  chain  defined  in  the  TDP  setup  window  which  SPEAR  has  executed  is 
displayed.  SPEAR  provides  the  analyst  with  a  useful  feature  to  display  data  before  and  after  each  selected  function  through  a 
simple  selected  function  through  a  simple  selection  of  the  processing  option  from  the  algorithm  list.  The  TDP  graphical 
results  for  a  staring  sensor  are  shown  in  Figures  9,  10,  11.  In  Figure  9  the  results  of  the  TDP  processing  chain  of  a  sensor 
viewing  4  targets  (two  of  which  are  CSOs)  with  white  noise  background  modeled.  For  this  case  the  TDP  algorithm  chain 
consists  of  a  saturation  test  (  monitoring  for  pixels  out  of  range,  essentially  the  virtual  FPA  in  counts,  output  from  the  ASP), 
the  Matched  Filter  (3x3  convolution  mask),  and  the  detection  thresholding.  Similar  TDP  results  are  shown  in  Figure  10  for  a 
IR  staring  sensor  viewing  the  a  similar  set  of  targets  against  a  SSGM  background  scene  (terrain  plus  cloud).  In  this  case  the 
analyst  can  choose  to  generate  the  targets  within  SSGM  for  the  specific  viewing  geometry  or  can  use  SPEAR  to  splice  the 
targets  into  regions  of  interest  within  the  SSGM  background  scene.  The  TDP  plots  can  be  rotated  for  optimum  viewing  and 
the  color  palette  can  be  modified  with  a  slider  bar  to  enhance  details.  The  ability  to  see  the  effects  of  each  function  is  another 
key  benefit  to  productivity  that  SPEAR  brings  to  data  analysis. 

Another  valuable  graphical  analysis  tool  is  the  Pulse  Matcher  window.  The  raw  extracted  data  and  the  best  fit  to 
the  3-D  data  that  was  achieved  is  displayed  in  Figure  11,  for  the  two  CSC  targets  in  this  case.  Also  displayed  within  the 
pulse  matcher  window  are  the  extracted  packet  statistics  such  as  size,  sum  squared  fit  errors  for  singles  and  duals,  the  Aikaike 
Information  Criterion  (AIC)  results  for  singles  and  duals,  the  number  of  targets  determined  per  packet,  and  finally  the  pulse 
matcher  FOM.  These  analysis  features  help  in  separating  data  quality  issues  from  algorithm  performance  issues. 

The  last  tool  in  the  Mission  Mode  analysis  toolbox  is  the  OSM  window.  Plotted  in  this  window  are  the  true 
positions  of  each  target,  SPEAR’ s  estimated  position,  and  the  physical  boundaries  of  the  extracted  data.  Any  spatial  biases 
introduced  by  the  signal  processor  are  readily  apparent.  A  quick  assessment  of  CSC  resolution  can  be  performed.  From  this 
plot,  the  analyst  quickly  sees  the  overall  estimation  performance  of  the  algorithm  chain. 
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FIGURE  9.  TDP  FUNCTIONAL  ANALYSIS  (STARING  SENSOR) 
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HGURE  10.  TDP FUNCTIONAL  ANALYSIS  (STARING  SENSOR  w/  SSGM  BACKGROUND) 
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FIGURE  11.  PULSE  MATCHER  ANALYSIS  WINDOW 


3.2  Statistical  Analysis  Mode 

In  the  Statistical  Analysis  Mode  the  analyst  accesses  to  the  Monte  Carlo  analysis  window.  In  this  window  the  user 
organizes  the  data  to  summarize  the  performance.  The  analyst  selects  from  a  list  the  data  sets  to  plot  and  the  measurement 
type  desired.  The  list  of  measurements  includes  radiometric  error,  angular  error,  pulse  matcher  figure  of  merit,  and  detection 
/  false  alarm  performance.  The  data  can  include  all  detection’s  or  only  those  SPEAR  classified  as  Singles  or  CSOs.  After  the 
desired  data  is  chosen,  the  analyst  sets  the  plot  parameters:  plot  title,  axis  titles,  and  automatic  user  scaling.  SPEAR  then 
plots  the  data  to  the  screen.  The  analyst  can  then  send  the  plots  to  a  printer,  to  an  encapsulated  PostScript  file  (EPSF)  or  save 
it  as  a  Graphics  Interchange  Format  (GIF)  file  for  later  use. 

Example  Statistical  Analysis  Mode  results  are  shown  in  Figure  12.  These  results  were  generated  by  modeling  a 
staring  sensor  viewing  6  targets  varying  in  SNR  from  2  to  64  with  the  noise  term  "white  noise"  and  the  target- to-detector 
phasing  options  turned  "on",  with  the  signal  processing  chain  consisting  of  a  matched  filter  and  detection  thresholding.  The 
results  show  the  percent  targets  detected,  RMP,  and  AMP  as  a  function  of  the  SNR.  As  the  SNR  gets  smaller  the  percent 
detected  drops  off  until  no  targets  are  detected  at  SNR  of  2,  and  the  corresponding  precisions  (RMP  and  AMP)  get  larger  as 
expected.  The  analyst  can  investigate  ways  to  improve  the  performance  of  the  sensor  and  signal  processor  by  modifying 
sensor  parameters  or  by  applying  other  signal  processing  filtering  and  thresholding  functions. 
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FIGURE  12.  MONTE  CARLO  ANALYSIS  RESULTS 

4.  SUMMARY 


This  brief  paper  can  only  highlight  some  of  SPEAR’ s  capabilities.  SPEAR  has  been  carefully  designed  to  fulfill  its 
dual  roles  of  IR  sensor  model  /  signal  processing  testbed  and  data  analysis  tool.  Extensive  documentation  has  been  produced 
covering  the  User  Interface  and  the  Application  Components,  including  guidelines  for  adding  new  algorithms  and  features. 

SPEAR  has  been  used  to  analyze  IR  sensor  test  /  flight  data  and  simulated  data  for  several  actual  and  proposed 
optical  systems.  Its  output  has  been  carefully  compared  to  laboratory  data  and  was  found  to  agree  within  the  uncertainties  of 
the  lab  setup.  Its  development  and  use  is  continuing  under  support  and  guidance  from  ASTP,  DITP,  and  GBI  program 
offices. 


Current  efforts  include  preflight  predictions  and  processing  of  actual  sensor  flight  data.  Another  major  emphasis  is 
the  continued  capability  to  interface  to  the  Synthetic  Scene  Generation  Model  (SSGM),  to  support  technology  assessments 
against  realistic  backgrounds  and  targets.  Through  this  interface  SPEAR  will  provide  sensor  modeling  for  the  ASTP  and 
DITP  Modeling  and  Simulation  Testbeds  being  used  to  evaluate  advanced  sensor  technologies  and  data  fusion  algorithms. 


Special  acknowledgments  go  out  to  the  NRC  Management  Staff  including  Kathy  Byrd,  Mike  Savage,  and  Randy 
Ormond  for  providing  the  opportunity  and  support  to  work  on  SPEAR  and  this  paper.  Special  consideration  is  given  to  the 


NRC  Technical  Staff  including  Dr.  Lary  Pinkley,  Dr.  Michael  Dechaine,  and  Dr.  Bifford  Lyons  for  their  guidance,  support, 
and  review  of  this  paper  and  their  continued  support  of  the  development  of  SPEAR  . 


